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(54) Automatic white balance detection and correction of an image captured in a digital camera 



(57) A method and apparatus for performing color 
balance enhancement using either a host or a device 
internal to the camera to adjust the color of a digital im- 



age taken by a digital camera. Digital image data repre- 
senting the image is analyzed to determine the amount 
of adjustment required to achieve proper white balance 
for the image. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the invention 

This invention relates to digital cameras, and more 
particularly to a method and apparatus for performing 
color adjustment upon images taken with a digital cam- 
era. 

2. Description of related Art 

Digital still cameras are being developed today to 
allow an image to be captured, digitized, stored, and re- 
produced using conventional printers coupled to a con- 
ventional personal computer. In most cases, an array 
ofcharge-coupled-device (CCD) detectors are used to 
capture the image. Each detector of the CCD array is 
used to capture one portion of the image. When light is 
received by a CCD detector, an electrical charge repre- 
sentative of the intensity of the light to which the detector 
has been exposed is accumulated. This charge can be 
coupled to an A/D (analog to digital) converter. The AID 
convertor produces a digital value that represents the 
charge on each of the detectors, and thus the intensity 
of the light received by each detector. 

If a color image is to be captured, then color filters 
are placed before each detector of the CCD array. Ac- 
cordingly, a detector which is associated with a filter that 
passes essentially only red light will form a red detector. 
The color detectors are frequently arranged in groups. 
For example, in one well known arrangement, two green 
detectors, a blue detector, and a red detector form a two 
by two grouping. This grouping is repeated throughout 
the array of detectors. By taking the digital value asso- 
ciated with each detector, the color composition of the 
image is captured. 

One problem that is encountered is that the amount 
of color detected by each detector may be altered due 
to the composition of the spectrum of light emitted by 
the particular light source which illuminates the image. 
For example, if an image is illuminated by a tungsten 
light source, then the image will be shifted toward the 
red spectrum, since tungsten light sources emit more 
red light than blue or green light. This shift will give the 
resulting photograph an undesirable reddish/orangish 
appearance. 

In accordance with one method that has been used 
to process an image to adjust for differences in the na- 
ture of the light, variable gain amplifiers are provided in 
line with the red and blue signals. The amount of gain 
applied to the red and blue signals is adjusted to com- 
pensate for the type of light which illuminates the image. 
A manual control on the camera allows the user to select 
between tungsten mode and daylight mode. In tungsten 
mode, the gains of the amplifiers that are responsible 
for amplifying the output from the red and blue detectors 



are set to a first gain ratio to compensate for the blue 
shift. In daylight mode, a second gain ratio is used which 
causes the gain of the amplifiers responsible for ampli- 
fying the red and blue signals to be approximately equal 
to the amount of gain which is set for the green signals. 

Selection of the particular gain ratio is also depend- 
ent upon the state of a relay. The relay overrides the 
manual control when a flash has been charged, since 
the use of a flash will alter the color composition of the 
image. That is, the gain ratio of the color signal amplifi- 
ers should be essentially the same when capturing an 
image using a flash device and when capturing an im- 
age during daylight. Therefore, when a flash device is 
charged to illuminate an image, the relay sets the mode 
of the variable gain amplifiers to the daylight setting, as 
is appropriate in the case in which a flash is to be used. 
Thus, by altering the gain of the amplifiers responsible 
for amplifying the outputs from the red and blue detec- 
tors, the color is properly balanced for the case in which 
an image is to be captured in tungsten lighting, daylight, 
or when a flash is used. 

However, this system can improperly adjust the 
color of the image that is captured if the flash is charged, 
but not actually activated. Furthermore, the amount of 
color adjustment that is preformed is essentially fixed at 
one of two particular ratios (i.e. , the gain of the amplifiers 
is set to one of two levels). Accordingly, if the amount of 
color adjustment that is required is other than that which 
would be desirable if the image were illuminated solely 
by tungsten or solely by natural sunlight, then the color 
of the image will be improperly adjusted. Still further, not 
every tungsten light source will have exactly the same 
color temperature (spectral composition of wave- 
lengths). 

A common way in which digital cameras have at- 
tempted to white balance an image requires that a pre- 
view image be taken. The preview image is used to de- 
termine the amount of adjustment that is required in or- 
der to white balance the image. There are several his- 
torical reasons why a preview image is used to deter- 
mine the amount of adj ustment that is required to white 
balance an image that is taken some time later. For ex- 
ample, in the case in which the adjustment is made by 
adjusting the gain of the amplifiers to which the output 
from the CCD array is coupled, the white balance infor- 
mation must be known at the time the image is read from 
the CCD array. 

However, the adjustment will be incorrect if the illu- 
mination of the image differs between the time the pre- 
view image is taken and the time the final image is taken. 
For example, if the preview image is taken without a 
flash, and the final image is taken with a flash, then the 
correction that is made to the final image based upon 
the characteristics of the preview image will be incorrect. 
A number of other situations can arise which would 
cause the illumination of the preview image to differ from 
the illumination of the final image. For example, the sun 
may become obscured by a cloud, a light may be turned 
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on or off. etc. 

Accordingly, it would be desirable to provide a sys- 
tem wherein the color adjustment is determined in a way 
that accurately reflects the characteristics of the final im- 
age. Furthermore, it would be desirable to provide a sys- s 
tern wherein the amount of color adjustment can be ad- 
justed to compensate for several different conditions 
and combinations of light sources. 

SUMMARY OF THE INVENTION 

The present invention is a method and apparatus 
for performing color balance of an image taken with a 
digital camera using either an host computer that is ex- 
ternal to the camera or in a device internal to the camera 
using the image data of the image to be white balance 
adjusted in order to determine the amount of adjustment 
required.. 

In accordance with the present invention, a digital 
image is captured by: first flushing, at a first point in time, 
any charge which has accumulated on a CCD light de- 
tector array; allowing light to strike the CCD detector af- 
ter the detector has been flushed; and determining the 
amount of charge that has accumulated upon the detec- 
tors that comprise the array. Once the image has been 
captured, a determination must be made as to whether 
the image requires white balancing. White balancing is 
typically required if the light source that illuminates ob- 
jects to be imaged has a "color temperature" that would 
cause the image to be tinted or biased in favor of, or 
away from, one color. For example, a tungsten light 
source will typically be biased in favor red. Therefore, in 
order for the image to be properly white balanced, the 
red component of each pixel must be adjusted with re- 
spect to the blue and green components of each pixel. 

In accordance with the present invention, the 
amount of the adjustment which is to be made is deter- 
mined by analysis of the image data to be adjusted (i. 
e., the "picture image"). In accordance with one embod- 
iment of the present invention, image pixels within the 
picture image are mapped into a two dimensional space, 
the two dimensions of this space being red/blue and 
green/blue. Image pixels that make up a reference im- 
age that was illuminated by a reference light source are 
then plotted in the two dimensional space. Each of the 
points at the periphery of the picture image are mapped 
into points at the periphery of the reference image by a 
transform. The transforms that are used to map the 
points in the picture image to the points in the reference 
image are then plotted in a transform space. A shape is 
defined in the transform space by the set of transforms 
associated with each of the points on the periphery of 
the picture image. Accordingly, a set of such shapes in 
the transform space will be defined for the picture image. 
The intersection of these shapes is then used to deter- 
mine source of light that illuminates the picture image 
and the adjustment factors required to properly white 
balance the picture image. 



It will be clear that since the image data to be ad- 
justed is used to determine the amount of the adjust- 
ment that is required, the adjustment will be more pre- 
cise than would be the case if a preview image is taken 
to determine how to adjust the picture image. 

In accordance with one embodiment of the present 
invention, the analysis and adjustment ofthe image data 
is performed in the camera. However, in an alternative 
embodiment of the present invention, the adjustment is 
performed outside the camera by a host device, such 
as a personal computer or printer. 

BRIEF DESCRIPTION OF THE DRAWING 

The objects, advantages, and features of this inven-. 
tion will become readily apparent in view of the following 
description, when read in conjunction with the accom- 
panying drawing, in which: 

Fig. 1 is a block diagram of a digital camera in ac- 
cordance with the present invention. 

Like reference numbers and designations in the 
various drawings refer to like elements. 

DETAILED DESCRIPTION OF THE INVENTION 

Throughout this description, the preferred embodi- 
ment and examples shown should be considered as ex- 
emplars, ratherthan limitationson the present invention. 

The present invention is a method and apparatus 
for performing automatic white balance on an image 
captured by a digital camera based upon characteristics 
of a light source as determined from an analysis of the 
image data. 

Figure 1 is a block diagram of one embodiment of 
a digital camera 100 in accordance with the present in- 
vention. The digital camera 1 00 includes: an optical lens 
101; an array of charge-coupled-device (CCD) detec- 
tors 103, including an array of color filters, each filter 
associated with a detector; a gain control circuit 105 ; 
an A/D (analog to digital) convertor 1 07; a control device 
109 (such as a microprocessor), preferably including a 
JPEG file format conversion device 1 1 5 and an EXI F file 
format device 117; a buffer memory 111; an image 
processing device 113, such as a DSP (digital signal 
processor); an image memory device 119; and an output 
port 120. 

Light enters the digital camera 100 through the lens 
101 and is focused on the array of CCD detectors 103. 
As is well known in the art, the color filters 102 associ- 
ated with each detector within the CCD array 1 03 cause 
the detectors to be sensitive to one particular color. In 
accordance with one embodiment of the present inven- 
tion, the CCD detectors are configured in a repeating 
pattern of two by two groups in which the top right de- 
tector is sensitive to red light, the top left detector is sen- 
sitive to blue green light, the bottom right detector is sen- 
sitive to green light, and the bottom left detector is sen- 
sitive to blue light. It should be understood that the array 
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of detectors may be fabricated such that the particular 
detectors are sensitive to other colors and are arranged 
in other configurations. 

Each detector accumulates a charge that is repre- 
sentative of the amount of light in one CCD pixel. A tim- s 
ing generator 104 is coupled to the CCD detector array 
103. The timing generator 104 controls the reading of 
the CCD detector array 103 in essentially conventional 
fashion. The timing generator 104 causes the charge 
accumulated by each detector to be serially applied to 10 
the input of the gain control device 105. The control de- 
vice 1 09 is coupled to the timing generator and controls 
the operation of the timing generator 104 in essentially 
conventional fashion. 

The timing generator 104 is capable of flushing the is 
CCD array 1 03 to essentially discharge any charge pre- 
viously accumulated the detectors within the array. 
Once the CCD array 103 has been flushed, the light that 
passes through the lens and the color filters causes 
charge to accumulate on each of the detectors within 20 
the CCD array 103 in an amount that is proportional to 
the amount of light that strikes each detector. The 
charge that accumulates on each detector within the 
CCD array 1 03 results in a voltage that is coupled to the 
gain control circuit 105. The gain control circuit 105 in- 25 
eludes an automatic gain control circuit and circuitry to 
implement a correlated double sampling process. The 
correlated double sampling piocess is a conventional 
process which accounts for overshoot and undershoot 
in the output from the CCD detectors as the CCD array 30 
is read. 

The control device 1 09 is coupled to the gain control 
device and controls the automatic gain control circuit. 
The output from the gain control circuit 105 is coupled 
to the A/D converter 107. The AID converter 107 con- 35 
verts to digital values the voltages read from the CCD 
detectors and processed by the gain control circuit 105. 
The resulting digital values are preferably directly pro- 
portional to the amount of light detected by each detec- 
tor. 40 

The A/D converter 107 is coupled to the DSP 113. 
The DSP 113 processes the color information stored in 
the buffer memory 111 to provide a demosaic function, 
to perform automatic white balance, and to sharpen the 
image. The DSP 1 1 3 determines which values are cou- 45 
pled from the A/D converter 1 07 to the DSP 1 1 3 are as- 
sociated with which detectors by the order in which the 
values are received from the A/D converter 107. The 
demosaic function and the automatic white balance 
functions are conventional functions. For example, in so 
accordance with one embodiment of the present inven- 
tion, the DSP 113 is a Part No. HD49811TFA commer- 
cially available from Hitachi and the automatic white bal- 
ance function is provided in the HD4981 1 TFA data sheet 
distributed by Hitachi. The output from the DSP 113 is 55 
a set of image pixels, each of which represent the color 
of a particular portion of the image that was captured by 
the CCD detector array 103. Accordingly, the complete 
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set of image pixels comprise the image data. The image 
data is stored in a buffer memory 111. The buffer mem- 
ory 111 is read by the control device 109. 

In accordance with one embodiment of the present 
invention, the control device 109 includes a convention- 
al dedicated integrated circuit which performs a JPEG 
file format conversion. Therefore, in accordance with 
one embodiment of the present invention, a combination 
of hardware and software executed within the control 
device 109 operate together to form a JPEG file format 
conversion device 1 1 5. However, in accordance with an- 
other embodiment of the present invention, the JPEG 
file format conversion device comprises only hardware. 
Alternatively, the JPEG file format conversion device 
115 is a software routine performed within a program- 
mable control device 1 09. In either case, file format con- 
version is performed in conventional fashion. In yet an- 
other embodiment of the present invention, no compres- 
sion is performed. In still another embodiment, com- 
pression is performed in accordance with a standard 
other than JPEG. 

The JPEG file format conversion process com- 
presses the information output from the DSP 113 and 
stored in the buffer 111 in accordance with the well 
known JPEG data compression standard. The informa- 
tion which is in JPEG format is then embedded within a 
file which conforms to "Digital Still Camera Image File 
Format Standard" (Version 1.0. July 13. 1995) (com- 
monly known as "EXIF"), defined by the JEIDA Electron- 
ic Still Camera Technical Committee. The file is embed- 
ded within the EXIF file by an EXIF file format device 
117 within the control device 109. In accordance with 
one embodiment of the present invention, the EXIF file 
format device is a combination of software executed 
within the control device 109 and dedicated hardware. 

However, in an alternative embodiment, the EXIF 
file format device 117 is a software routine performed 
within the control device 109. In yet another embodi- 
ment of the present invention, the EXIF file format de- 
vice 117 comprises only dedicated hardware. 

In the preferred embodiment of the present inven- 
tion, the image information is compressed in accord- 
ance with the well known JPEG data compression tech- 
nique and presented in YC b C r format with 4:2:2 subsam- 
pling. The EXIF file is then transferred to a host for fur- 
ther processing. However, it should be understood that 
in accordance with an alternative embodiment of the 
present invention, the output from the DSP 113 may be 
formatted in any manner which allows a host 121 , such 
as a personal computer or printer, to read and interpret 
the information provided by the camera 100 to the host 
121 . In accordance with one embodiment of the present 
invention, the image data is not compressed. 

In accordance with the present invention, the image 
data is analyzed to determine how to adjust the digital 
values in the image data to correct for differences in the 
characteristics of the light source used to illuminate the 
image. In accordance with one embodiment of the 
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present invention, the image data is analyzed to deter- 
mine the amount of adjustment to be applied to the im- 
age data to properly white balance the image. The anal- 
ysis involves mathematical manipulation of the image 
data which can be performed by a programmable de- 
vice, such as a microprocessor or DSP either within or 
external to the camera (e.g., in the host). In accordance 
with one embodiment of the present invention, the im- 
age data is analyzed with respect to a reference image. 
The reference image preferably has an essentially full 
spectrum of visible wavelengths and is illuminated by a 
reference light source. The image data for the reference 
image is mapped into a two dimensional space in which 
the horizontal axis is red divided by blue and the vertical 
axis is green divided by blue. This mapping reduces the 
three dimensional (rgb) points to chromaticity defined 
by two values (i.e., the red/blue value and the green/ 
blue value). Accordingly, a gamut of points can be plot- 
ted in this two dimensional space, each point represent- 
ing the color of one pixel in the reference image. A set 
of points which, when connected from one point to the 
next, define the outer most boundary of this reference 
gamut are identified. These peripheral points are then 
saved. 

Once the peripheral points of the reference image 
have been established, a picture image can be analyzed 
with respect to the reference image by first mapping the 
picture image into the two-dimensional space such that 
each pixel defines a point in the two-dimensional space. 
Then peripheral points for the picture image are deter- 
mined. Each one of these points are than mapped to 
one of the peripheral points of the reference image. The 
transform that is used to map the peripheral point of the 
picture image to the peripheral point of the reference im- 
age is then defined by two values. The first value is the 
quotient of the red/blue values, and the second value is 
the quotient of the green/blue values. Each set of trans- 
forms associated with one point of the picture image and 
mapped to each point in the reference image will define 
a shape. Therefore, a set of shapes will be defined by 
mapping each of the points in the picture image to each 
of the points in the reference image. The intersection of 
these shapes defines a new shape. This new shape can 
then be analyzed to determine the light source which 
most likely was used to illuminate the scene. 

It should be understood that in an alternative em- 
bodiment of the present invention, the image data may 
be analyzed by any other methods for determining the 
adjustment required to color balance the image. 

In another embodiment of the present invention, the 
device for analyzing the image data is provided within 
the camera. For example, in one embodiment of the 
present invention, the DSP 11 3 performs the analysis of 
the image data. As a result of the analysis of the image 
data, the color of the image pixels that make-up the im- 
age are adjusted to achieve proper white balance. It 
should be understood that the image data may not be 
compressed, or the compression may be applied after 



either analysis of the image data or after the adjust- 
ments to the color balance are made. 

Summary 

s 

A number of embodiments of the present invention 
have been described. Nevertheless, it will be under- 
stood that various modifications may be made without 
departing from the spirit and scope of the invention. Fur- 
to thermore, the components of the digital camera are de- 
scribed as one example of devices which perform par- 
ticular functions within a digital camera. Accordingly, the 
functions which are described as being performed within 
the digital camera may be performed by any device that 
is is capable of performing the particular functions. For ex- 
ample, the control device 1 09 may be a microprocessor, 
an application specific integrated circuit, a discrete cir- 
cuit, a state machine, or any other circuit or component 
that is capable of performing the functions that are de- 
20 scribed above. Accordingly, it is to be understood that 
the invention is not to be limited by the specific illustrated 
embodiments, but only by the scope of the appended 
claims. 

25 

Claims 

1. A digital camera (100), comprising: 

30 an image sensing arrangement (101, 102, 103, 

104, 105, 107) for capturing an image and its 
color composition resulting from a source of im- 
age illuminating light; 

35 an image storage arrangement (111, 113, 115, 

1 1 7) responsive to said image sensing arrange- 
ment for storing non-color-corrected data indic- 
ative of said captured image; and 

40 a color compensation arrangement (109, 119, 

120, 121, 123) for analyzing the stored non- 
color-corrected data to define a set of transfor- 
mation values that enable the non-color-cor- 
rected data to be automatically color balanced 

45 without using a pre-exposure scene sampling. 

2. A digital camera (100) according to claim 1 , wherein 
said image sensing arrangement (101, 102, 103, 
104, 105, 107) includes: 

so 

a sensor array (103) for converting light into 
electrical energy; 

an array of color filters (102) for causing said 
ss sensor array (103) to be sensitive to certain 

specific color spectrums; and 

an optical lens (1 01 ) for directing light from an 
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illuminated image onto said array of color filters 
(102) to facilitate capturing the color compen- 
sation of said image when illuminated by said 
source of light. 

3. Adigitalcamera(100)accordingtoclaim2, wherein 
said sensor array (103) is an array of charge cou- 
pled devices. 

4. A digital camera (100) according to claims 2 or 3, 
wherein said sensing arrangement further includes: 

a timing generator (1 04) coupled to said sensor 
array (103) for facilitating the sequential read- 
ing of discrete pixel data values; 

a gain control circuit (1 05) coupled to said sen- 
sor array (103) for correcting overshoot and un- 
dershoot signal anomalies in processing the 
captured image from said sensor array (1 03) in- 
to the discrete pixel data values; and 

an analog to digital converter (107) for convert- 
ing each discrete pixel data value into a dis- 
crete digital value indicative of one pixel of non- 
color-corrected data. 



ancing is required to compensate for color shift- 
ing in the captured image resulting from the 
spectral characteristics of said source of light. 

5 7. A digital camera (100) according to any one of the 
preceding claims, wherein said color compensation 
arrangement (109, 119, 120, 121, 123) includes: 

a digital processing apparatus (1 1 9) for execut- 
10 ing a control program; and 

an image memory arrangement (109) for for- 
matting and converting the non-color-corrected 
data. 

15 

8. A digital camera (100) according to claim 7, wherein 
said control program causes said digital processing 
apparatus to automatically adjust the color balance 
of said captured image, said digital processing ap- 

20 paratus including means responsive to said control 
program to cause the non-color-corrected data to 
be analyzed with respect to a reference image to 
determine whether individual ones of the pixel data 
values need to be adjusted to achieve a proper 

25 white balance relative to said source of light causing 
the illumination of the captured image. 



5. A digital camera (100) according to any one of the 
preceding claims, wherein said color compensation 
arrangement (109, 119, 120, 121, 123) includes: 30 

a processing circuit (109, 119) for analyzing 
each discrete digital value indicative of a pixel 
of non -color-corrected data has a sufficient 
color shift due to the spectral characteristics of 35 
said source of light to require white balance ad- 
justment; 

a control device (120, 121, 123) responsive to 
said processing circuit (119) for adjusting indi- *o 
vidual ones of the pixels of non-color-corrected 
data with sufficient color shift to require white 
balance correction. 

6. A digital camera (100) according to any one of the 45 
preceding claims, wherein said color compensation 
arrangement (109, 119, 120, 121, 123) includes: 

a host arrangement (120, 121, 123) having: 

so 

an analysis circuit for analyzing the non-color- 
corrected data to determine whether it needs 
color balancing due to the spectral characteris- 
tics of said source of light; and 

55 

a control device responsive to the analysis cir- 
cuit for adjusting the non-color-corrected data 
when a determination is made that color bal- 



9. A digital camera (100) according to claim 8, wherein 
digital processing apparatus includes: 

two-dimensional space means responsive to 
said control program for mapping image data 
for said reference image into two dimensional 
color space having a horizontal axis reference 
indicative of the red color spectrum divided by 
the blue color spectrum and a vertical axis ref- 
erence indicative of the green color spectrum 
divided by the blue color spectrum; 

wherein said reference image comprises a set 
of digital pixel values of an object illuminated 
by a reference light source that includes sub- 
stantially the full spectrum of visible wave- 
lengths of light; and 

wherein said two-dimensional space means 
transforms three-dimensional chromaticity 
space points into two-dimensional color space 
points of red divided by blue digital values and 
green divided by blue digital values, each point 
in the two-dimensional color space being indic- 
ative of the color of one pixel in said reference 
image; and 

wherein the outer most boundary values of the 
two-dimensional color space defining the refer- 
ence image define a reference gamut, said ref- 
erence gamut being stored to facilitate color 
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balancing image transformation purposes; and justed relative to said reference light source. 



wherein said two-dimensional space means is 
further responsive to said control program for 
mapping the non-color-corrected data indica- $ 
tive of the captured image into the two-dimen- 
sional color space to define each pixel value in 
the captured image; and 

wherein the outer most boundary values of the 10 
two-dimensional color space defining the cap- 
tured image define a non-color-corrected gam- 
ut referenced to said reference gamut; and 

transformation means responsive to said con- is 
trol program for establishing a set of transfor- 
mation values for each pixel of the captured im- 
age, each one of set of transformation values 
including two discrete values, one value being 
the quotient value of the red spectrum compo- 20 
nent of the pixel divided by blue spectrum com- 
ponent of the pixel and the other value being 
the quotient value of the green spectrum com- 
ponent of the pixel divided by the blue spectrum 
component of the pixel; ^ 5 

wherein each set of transformations associated 
with an individual pixel in the captured image is 
mapped to an associated pixel in the captured 
image to define a shape so that a set of inter- 30 
secting shapes are defined where the intersec- 
tion of the shapes define a new set of shapes 
that help facilitate determining the light source 
most likely utilized in illuminating a scene indic- 
ative of the captured image without using pre- 3$ 
exposure scene sampling. 

10. A system (100) for automatic white balance detec- 
tion and correction, comprising: 

40 

memory mean (101, 102, 103, 104, 105, 107, 
111, 115, 117, 119, 120, 123) for capturing and 
storing non-color-corrected data indicative of a 
captured image illuminated by a source of light, 
said captured image being defined by a plurality 
of individual pixels; 

determining means (113, 119) coupled to said 
memory means for analyzing the non-color- 
corrected data to determine the white balance so 
of each individual pixel in the captured image 
relative to a reference light source without us- 
ing a pre-exposure scene sample; and 

color balance means (121) responsive to said 55 
determining means for adjusting the white bal- 
ance of each individual pixel in the captured im- 
age so that white balance of each pixel is ad- 
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